Three-dimensional (3D) imaging is extremely popular in medical imaging as it enables diagnosis and disease monitoring through complete anatomical coverage. Computed Tomography or Magnetic Resonance Imaging (MRI) techniques are commonly used, however, anisotropic volumes with thick slices are often acquired to reduce scan times. Deep learning (DL) can be used to recover high-resolution features in the low-resolution dimension through super-resolution reconstruction (SRR). However, this often relies on paired training data which is unavailable in many medical applications. We describe a novel approach that only requires native anisotropic 3D medical images for training. This method relies on the observation that small 2D patches extracted from a 3D volume contain similar visual features, irrespective of their orientation. Therefore, it is possible to leverage disjoint patches from the high-resolution plane, to learn SRR in the low-resolution plane. Our proposed unpaired approach uses a modified CycleGAN architecture with a cycle-consistent gradient mapping loss: Cycle Loss Augmented Degradation Enhancement (CLADE). We show the feasibility of CLADE in an exemplar application; anisotropic 3D abdominal MRI data. We demonstrate superior quantitative image quality with CLADE over supervised learning and conventional CycleGAN architectures. CLADE also shows improvements over anisotopic volumes in terms of qualitative image ranking and quantitative edge sharpness and signal-to-noise ratio. This paper demonstrates the potential of using CLADE for super-resolution reconstruction of anisotropic 3D medical imaging data without the need for paired training data.

Time-resolved MR angiography (tMRA) has been widely used for dynamic contrast enhanced MRI (DCE-MRI) due to its highly accelerated acquisition. In tMRA, the periphery of the k-space data are sparsely sampled so that neighbouring frames can be merged to construct one temporal frame. However, this view-sharing scheme fundamentally limits the temporal resolution, and it is not possible to change the view-sharing number to achieve different spatio-temporal resolution trade-off. Although many deep learning approaches have been recently proposed for MR reconstruction from sparse samples, the existing approaches usually require matched fully sampled k-space reference data for supervised training, which is not suitable for tMRA. This is because high spatio-temporal resolution ground-truth images are not available for tMRA. To address this problem, here we propose a novel unsupervised deep learning using optimal transport driven cycle-consistent generative adversarial network (cycleGAN). In contrast to the conventional cycleGAN with two pairs of generator and discriminator, the new architecture requires just a single pair of generator and discriminator, which makes the training much simpler and improves the performance. Reconstruction results using in vivo tMRA data set confirm that the proposed method can immediately generate high quality reconstruction results at various choices of view-sharing numbers, allowing us to exploit better trade-off between spatial and temporal resolution in time-resolved MR angiography.

O PTIMAL T RANSPORT, C YCLEGAN, AND P ENALIZED LS FOR U NSUPERVISEDL EARNING IN I NVERSE P ROB-LEMS Byeongsu Sim 1 Gyutaek Oh 2 Sungjun Lim 2 Jong Chul Y e 1,2 1 Department of Mathematical Sciences, KAIST, Daejeon, Republic of Korea 2 Department of Bio and Brain Engineering, KAIST, Daejeon, Republic of Korea A BSTRACT The penalized least squares (PLS) is a classic approach to inverse problems, where a regularization term is added to stabilize the solution. Optimal transport (OT) is another mathematical framework for computer vision tasks by providing means to transport one measure to another at minimal cost. Cycle-consistent generative adversarial network (cycleGAN) is a recent extension of GAN to learn target distributions with less mode collapsing behavior. Although similar in that no supervised training is required, the algorithms look different, so the mathematical relationship between these approaches is not clear. In this article, we provide an important advance to unveil the missing link. Specifically, we reveal that a cycle-GAN architecture can be derived as a dual formulation of the optimal transport problem, if the PLS with a deep learning penalty is used as a transport cost between the two probability measures from measurements and unknown images. This suggests that cycleGAN can be considered as stochastic generalization of classical PLS approaches. Our derivation is so general that various types of cy-cleGAN architecture can be easily derived by merely changing the transport cost.